{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# Cloud Computing with Google Earth Engine and GeoAI\n",
    "\n",
    "[![image](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/gee-community/geemap/blob/master/docs/workshops/GeoAI_2024.ipynb)\n",
    "\n",
    "This notebook is designed for the workshop presented at the [Symposium on Spatiotemporal Data Science](https://sdl.gis.harvard.edu/event/symposium-spatiotemporal-data-science-geoai-social-sciences).\n",
    "\n",
    "The packages used in this notebook include:\n",
    "\n",
    "- [earthengine-api](https://earthengine.google.com)\n",
    "- [geemap](https://geemap.org)\n",
    "- [samgeo](https://samgeo.gishub.org)\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "To use geemap and the Earth Engine Python API, you must [register](https://code.earthengine.google.com/register) for an Earth Engine account and follow the instructions [here](https://docs.google.com/document/d/1ZGSmrNm6_baqd8CHt33kIBWOlvkh-HLr46bODgJN1h0/edit?usp=sharing) to create a Cloud Project. Earth Engine is free for [noncommercial and research use](https://earthengine.google.com/noncommercial). To test whether you can use authenticate the Earth Engine Python API, please run [this notebook](https://colab.research.google.com/github/gee-community/geemap/blob/master/docs/notebooks/geemap_colab.ipynb) on Google Colab.\n",
    "\n",
    "## Use Colab GPU runtime\n",
    "\n",
    "Runtime -> Change runtime type -> Hardware accelerator -> T4GPU\n",
    "\n",
    "![](https://i.imgur.com/MwpD9Rt.png)\n",
    "\n",
    "## Install dependencies\n",
    "\n",
    "Uncomment and run the following cell to install the required dependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %pip install -U geemap segment-geospatial groundingdino-py leafmap localtileserver pycrs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2",
   "metadata": {},
   "source": [
    "## Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import ee\n",
    "import geemap\n",
    "import leafmap\n",
    "from samgeo import SamGeo\n",
    "from samgeo.text_sam import LangSAM"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4",
   "metadata": {},
   "source": [
    "## Authenticate and initialize Earth Engine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5",
   "metadata": {},
   "outputs": [],
   "source": [
    "ee.Authenticate()\n",
    "ee.Initialize(project=\"YOUR-PROJECT-ID\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6",
   "metadata": {},
   "source": [
    "## Visualize Earth Engine data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7",
   "metadata": {},
   "outputs": [],
   "source": [
    "m = geemap.Map()\n",
    "point = ee.Geometry.Point(-122.259679, 37.871838)\n",
    "collection = (\n",
    "    ee.ImageCollection(\"USDA/NAIP/DOQQ\")\n",
    "    .filterBounds(point)\n",
    "    .filterDate(\"2008-01-01\", \"2018-01-01\")\n",
    "    .filter(ee.Filter.listContains(\"system:band_names\", \"N\"))\n",
    ")\n",
    "image = collection.first()\n",
    "m.add_layer(image, {}, \"NAIP\")\n",
    "m.center_object(point, 16)\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8",
   "metadata": {},
   "source": [
    "## Download Earth Engine data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9",
   "metadata": {},
   "outputs": [],
   "source": [
    "bbox = m.user_roi_coords()\n",
    "if bbox is None:\n",
    "    bbox = [-122.2666, 37.8682, -122.252, 37.8752]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10",
   "metadata": {},
   "outputs": [],
   "source": [
    "geemap.ee_to_geotiff(\n",
    "    image, \"naip.tif\", bbox, zoom=17, vis_params={\"bands\": [\"R\", \"G\", \"B\"]}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11",
   "metadata": {},
   "source": [
    "## Automatic segmentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12",
   "metadata": {},
   "outputs": [],
   "source": [
    "sam = SamGeo(\n",
    "    model_type=\"vit_h\",\n",
    "    device=None,\n",
    "    sam_kwargs=None,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13",
   "metadata": {},
   "outputs": [],
   "source": [
    "sam.generate(\"naip.tif\", output=\"masks.tif\", foreground=True, unique=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14",
   "metadata": {},
   "source": [
    "## Visualize segmentation results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15",
   "metadata": {},
   "outputs": [],
   "source": [
    "sam.show_masks(cmap=\"binary_r\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16",
   "metadata": {},
   "outputs": [],
   "source": [
    "sam.show_anns(axis=\"off\", alpha=1, output=\"annotations.tif\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17",
   "metadata": {},
   "outputs": [],
   "source": [
    "m.add_raster(\"annotations.tif\", opacity=0.5, layer_name=\"Masks\")\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "18",
   "metadata": {},
   "outputs": [],
   "source": [
    "sam.tiff_to_vector(\"masks.tif\", \"masks.shp\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19",
   "metadata": {},
   "outputs": [],
   "source": [
    "style = {\n",
    "    \"color\": \"#3388ff\",\n",
    "    \"weight\": 2,\n",
    "    \"fillColor\": \"#7c4185\",\n",
    "    \"fillOpacity\": 0.5,\n",
    "}\n",
    "m.add_vector(\"masks.shp\", layer_name=\"Vector\", style=style)\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20",
   "metadata": {},
   "source": [
    "## Download sample data\n",
    "\n",
    "### Create an interactive map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map(center=[40.427495, -86.913638], zoom=18, height=700)\n",
    "m.add_basemap(\"SATELLITE\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22",
   "metadata": {},
   "source": [
    "Pan and zoom the map to select the area of interest. Use the draw tools to draw a polygon or rectangle on the map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "if m.user_roi_bounds() is not None:\n",
    "    bbox = m.user_roi_bounds()\n",
    "else:\n",
    "    bbox = [-86.9167, 40.4262, -86.9105, 40.4289]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24",
   "metadata": {},
   "source": [
    "### Download map tiles\n",
    "\n",
    "Download maps tiles and mosaic them into a single GeoTIFF file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "image = \"image.tif\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26",
   "metadata": {},
   "source": [
    "Specify the basemap as the source."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "leafmap.map_tiles_to_geotiff(\n",
    "    output=image, bbox=bbox, zoom=18, source=\"Satellite\", overwrite=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28",
   "metadata": {},
   "source": [
    "You can also use your own image. Uncomment and run the following cell to use your own image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29",
   "metadata": {},
   "outputs": [],
   "source": [
    "# image = '/path/to/your/own/image.tif'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30",
   "metadata": {},
   "source": [
    "Display the downloaded image on the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m.layers[-1].visible = False  # turn off the basemap\n",
    "m.add_raster(image, layer_name=\"Image\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/YHwrpS2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33",
   "metadata": {},
   "source": [
    "## Automatic mask generation\n",
    "\n",
    "### Initialize SAM class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam = SamGeo(\n",
    "    model_type=\"vit_h\",\n",
    "    sam_kwargs=None,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35",
   "metadata": {},
   "source": [
    "### Automatic mask generation\n",
    "\n",
    "Segment the image and save the results to a GeoTIFF file. Set `unique=True` to assign a unique ID to each object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.generate(image, output=\"masks.tif\", foreground=True, unique=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.show_masks(cmap=\"binary_r\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/kWqLVuL.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39",
   "metadata": {},
   "source": [
    "Show the object annotations (objects with random color) on the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.show_anns(axis=\"off\", alpha=1, output=\"annotations.tif\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/J6Ie0Zj.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42",
   "metadata": {},
   "source": [
    "Compare images with a slider."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "leafmap.image_comparison(\n",
    "    \"image.tif\",\n",
    "    \"annotations.tif\",\n",
    "    label1=\"Satellite Image\",\n",
    "    label2=\"Image Segmentation\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/cm4QyaR.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45",
   "metadata": {},
   "source": [
    "Add image to the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "46",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m.add_raster(\"annotations.tif\", opacity=0.5, layer_name=\"Masks\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/Y6EaGVN.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48",
   "metadata": {},
   "source": [
    "Convert the object annotations to vector format, such as GeoPackage, Shapefile, or GeoJSON."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.raster_to_vector(\"masks.tif\", \"masks.shp\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m.add_vector(\"masks.shp\", layer_name=\"Masks vector\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/N0xVt9S.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52",
   "metadata": {},
   "source": [
    "### Automatic mask generation options\n",
    "\n",
    "There are several tunable parameters in automatic mask generation that control how densely points are sampled and what the thresholds are for removing low quality or duplicate masks. Additionally, generation can be automatically run on crops of the image to get improved performance on smaller objects, and post-processing can remove stray pixels and holes. Here is an example configuration that samples more masks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam_kwargs = {\n",
    "    \"points_per_side\": 32,\n",
    "    \"pred_iou_thresh\": 0.86,\n",
    "    \"stability_score_thresh\": 0.92,\n",
    "    \"crop_n_layers\": 1,\n",
    "    \"crop_n_points_downscale_factor\": 2,\n",
    "    \"min_mask_region_area\": 100,\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam = SamGeo(\n",
    "    model_type=\"vit_h\",\n",
    "    sam_kwargs=sam_kwargs,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.generate(image, output=\"masks2.tif\", foreground=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.show_masks(cmap=\"binary_r\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/S2LYen8.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.show_anns(axis=\"off\", opacity=1, output=\"annotations2.tif\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/opEKsUu.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60",
   "metadata": {},
   "source": [
    "Compare images with a slider."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "leafmap.image_comparison(\n",
    "    image,\n",
    "    \"annotations.tif\",\n",
    "    label1=\"Image\",\n",
    "    label2=\"Image Segmentation\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62",
   "metadata": {},
   "source": [
    "## Use points as input prompts\n",
    "\n",
    "### Initialize SAM class"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63",
   "metadata": {},
   "source": [
    "Set `automatic=False` to disable the `SamAutomaticMaskGenerator` and enable the `SamPredictor`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map(center=[40.427495, -86.913638], zoom=18, height=700)\n",
    "image = \"image.tif\"\n",
    "m.add_raster(image, layer_name=\"Image\")\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam = SamGeo(\n",
    "    model_type=\"vit_h\",\n",
    "    automatic=False,\n",
    "    sam_kwargs=None,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66",
   "metadata": {},
   "source": [
    "Specify the image to segment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.set_image(image)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68",
   "metadata": {},
   "source": [
    "### Image segmentation with input points\n",
    "\n",
    "A single point can be used to segment an object. The point can be specified as a tuple of (x, y), such as (col, row) or (lon, lat). The points can also be specified as a file path to a vector dataset. For non (col, row) input points, specify the `point_crs` parameter, which will automatically transform the points to the image column and row coordinates.\n",
    "\n",
    "Try a single point input:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "point_coords = [[-86.913162, 40.427157]]\n",
    "sam.predict(point_coords, point_labels=1, point_crs=\"EPSG:4326\", output=\"mask1.tif\")\n",
    "m.add_raster(\"mask1.tif\", layer_name=\"Mask1\", nodata=0, cmap=\"Blues\", opacity=1)\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/zUMLUsn.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71",
   "metadata": {},
   "source": [
    "Try multiple points input:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "point_coords = [\n",
    "    [-86.913162, 40.427157],\n",
    "    [-86.913425, 40.427157],\n",
    "    [-86.91343, 40.427721],\n",
    "    [-86.913012, 40.427741],\n",
    "]\n",
    "sam.predict(point_coords, point_labels=1, point_crs=\"EPSG:4326\", output=\"mask2.tif\")\n",
    "m.add_raster(\"mask2.tif\", layer_name=\"Mask2\", nodata=0, cmap=\"Greens\", opacity=1)\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/zUMLUsn.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74",
   "metadata": {},
   "source": [
    "### Interactive segmentation\n",
    "\n",
    "Display the interactive map and use the marker tool to draw points on the map. Then click on the `Segment` button to segment the objects. The results will be added to the map automatically. Click on the `Reset` button to clear the points and the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "75",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m = sam.show_map()\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/3W7JGqP.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77",
   "metadata": {},
   "source": [
    "## Bounding box input prompts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78",
   "metadata": {},
   "source": [
    "### Create an interactive map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "79",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map(center=[40.427495, -86.913638], zoom=18, height=700)\n",
    "image = \"image.tif\"\n",
    "m.add_raster(image, layer_name=\"Image\")\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam = SamGeo(\n",
    "    model_type=\"vit_h\",\n",
    "    automatic=False,\n",
    "    sam_kwargs=None,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81",
   "metadata": {},
   "source": [
    "Specify the image to segment. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.set_image(image)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83",
   "metadata": {},
   "source": [
    "### Create bounding boxes\n",
    "\n",
    "If no rectangles are drawn, the default bounding boxes will be used as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "if m.user_rois is not None:\n",
    "    boxes = m.user_rois\n",
    "else:\n",
    "    boxes = [\n",
    "        [-86.913654, 40.426967, -86.912774, 40.427881],\n",
    "        [-86.914780, 40.426256, -86.913997, 40.426852],\n",
    "        [-86.913632, 40.426215, -86.912581, 40.426820],\n",
    "    ]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85",
   "metadata": {},
   "source": [
    "## Segment the image\n",
    "\n",
    "Use the `predict()` method to segment the image with specified bounding boxes. The `boxes` parameter accepts a list of bounding box coordinates in the format of [[left, bottom, right, top], [left, bottom, right, top], ...], a GeoJSON dictionary, or a file path to a GeoJSON file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.predict(boxes=boxes, point_crs=\"EPSG:4326\", output=\"mask.tif\", dtype=\"uint8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87",
   "metadata": {},
   "source": [
    "## Display the result\n",
    "\n",
    "Add the segmented image to the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m.add_raster(\"mask.tif\", cmap=\"viridis\", nodata=0, opacity=0.6, layer_name=\"Mask\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/9y31xUH.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90",
   "metadata": {},
   "source": [
    "## Text promots\n",
    "\n",
    "### Initialize LangSAM class\n",
    "\n",
    "The initialization of the LangSAM class might take a few minutes. The initialization downloads the model weights and sets up the model for inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map(center=[40.427495, -86.913638], zoom=18, height=700)\n",
    "image = \"image.tif\"\n",
    "m.add_raster(image, layer_name=\"Image\")\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam = LangSAM()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93",
   "metadata": {},
   "source": [
    "### Specify text prompts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "text_prompt = \"tree\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95",
   "metadata": {},
   "source": [
    "### Segment the image\n",
    "\n",
    "Part of the model prediction includes setting appropriate thresholds for object detection and text association with the detected objects. These threshold values range from 0 to 1 and are set while calling the predict method of the LangSAM class.\n",
    "\n",
    "`box_threshold`: This value is used for object detection in the image. A higher value makes the model more selective, identifying only the most confident object instances, leading to fewer overall detections. A lower value, conversely, makes the model more tolerant, leading to increased detections, including potentially less confident ones.\n",
    "\n",
    "`text_threshold`: This value is used to associate the detected objects with the provided text prompt. A higher value requires a stronger association between the object and the text prompt, leading to more precise but potentially fewer associations. A lower value allows for looser associations, which could increase the number of associations but also introduce less precise matches.\n",
    "\n",
    "Remember to test different threshold values on your specific data. The optimal threshold can vary depending on the quality and nature of your images, as well as the specificity of your text prompts. Make sure to choose a balance that suits your requirements, whether that's precision or recall."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.predict(image, text_prompt, box_threshold=0.24, text_threshold=0.24)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97",
   "metadata": {},
   "source": [
    "### Visualize the results\n",
    "\n",
    "Show the result with bounding boxes on the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.show_anns(\n",
    "    cmap=\"Greens\",\n",
    "    box_color=\"red\",\n",
    "    title=\"Automatic Segmentation of Trees\",\n",
    "    blend=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/qRcy16Z.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "100",
   "metadata": {},
   "source": [
    "Show the result without bounding boxes on the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "101",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.show_anns(\n",
    "    cmap=\"Greens\",\n",
    "    add_boxes=False,\n",
    "    alpha=0.5,\n",
    "    title=\"Automatic Segmentation of Trees\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "102",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/TvqGByH.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "103",
   "metadata": {},
   "source": [
    "Show the result as a grayscale image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "104",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.show_anns(\n",
    "    cmap=\"Greys_r\",\n",
    "    add_boxes=False,\n",
    "    alpha=1,\n",
    "    title=\"Automatic Segmentation of Trees\",\n",
    "    blend=False,\n",
    "    output=\"trees.tif\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "105",
   "metadata": {},
   "source": [
    "Convert the result to a vector format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "106",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.raster_to_vector(\"trees.tif\", \"trees.shp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "107",
   "metadata": {},
   "source": [
    "Show the results on the interactive map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "108",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "m.add_raster(\"trees.tif\", layer_name=\"Trees\", palette=\"Greens\", opacity=0.5, nodata=0)\n",
    "style = {\n",
    "    \"color\": \"#3388ff\",\n",
    "    \"weight\": 2,\n",
    "    \"fillColor\": \"#7c4185\",\n",
    "    \"fillOpacity\": 0.5,\n",
    "}\n",
    "m.add_vector(\"trees.shp\", layer_name=\"Vector\", style=style)\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "109",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/WDQgECD.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "110",
   "metadata": {},
   "source": [
    "### Interactive segmentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "111",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sam.show_map()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "112",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/Zn7Dwty.png)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
